Competitive Benchmarking: Lessons Learned from the Trading Agent Competition

stimulated by well-designed competitions that tackle intriguing and often very complex problems. Chess, poker, stock trading, real-time strategy games, robot soccer, robot rescue or planning, and autonomous vehicles are among the most well known. Adaptability, proactiveness, and interoperability are essential characteristics of these games. Many of these competitions focus on development of rational behavior that would allow dominance over competitors. Apart from game play, though, one of the main reasons for these competitions is to bridge the gap between AI and real-life domains. In many real-life domains, such as trading environments, selfinterested entities need to operate subject to limited time and information. Additionally, the web has mediated an ever broader range of transactions, urging participants to concurrently trade across multiple markets. All these have generated the need for technologies that empower prompt investigation of large volumes of data and rapid evaluation of numerous alternative strategies in the face of constantly changing market conditions (Bichler, Gupta, and Ketter 2010). AI and machine-learning techniques, including neural networks and genetic algorithms, are continuously gaining ground in the support of such trading scenarios. User modeling, price forecasting, market equilibrium prediction, and strategy optimization are typical cases where AI typically provides reliable solutions. Yet, the adoption and deployment of AI practices in real trading environments remains limited, since the proprietary nature of markets precludes open benchmarking, which is critical for further scientific progress. To Competition Reports